P23: coverage after Tests (capability coverage, opt-in) - #39

P23, for Etappe 3 of «Funktionen». Decisions: all ★ except 2 (report only in the embed, marks only in the IDE) and the English full-coverage note as given.

What learners see

When all examples pass, the Tests report lists which lines of the tested file's functions no example executed, each with its code:

• de: «Von keinem Beispiel ausgeführt:» · «Zeile 11: return "ungenügend"», consecutive lines of one function as «Zeilen 16–18:» with the lines below
• en: «Not run by any example:» · «Line 11: …» / «Lines 16–18:»
• every line ran: «Jede Zeile der Funktionen wurde von mindestens einem Beispiel ausgeführt. Ob die Beispiele auch die Grenzfälle prüfen, zeigt das nicht.» / «Every line of the functions was run by at least one example. This doesn't show whether the examples also check the edge cases.»

The box is neutral (grey bar, not the green «bestanden» style). With a failing example, or a file that doesn't load, nothing about coverage shows.

In the IDE (with the editor) the same lines are also marked in the editor: a pale orange line background and a bar in the gutter, using the hint tokens (the editor's own background is --glp-surface). The marks go away on the first edit of that file, on Run, on the next Tests and on reset.

What counts (src/runtime/doctests.py)

• Statements in function bodies of the tested file, including methods and nested functions. Not counted: def lines, decorators, docstrings, class bodies, code outside functions, other files (libraries), statements without bytecode (global, while True).
• A statement counts as executed when any of its lines ran. For a compound statement only its header counts (if note >= 4:).
• Recorded with sys.monitoring (Python 3.12, Pyodide 0.26.2) during the examples only, so a top-level print(f(1)) doesn't count. Each line reports once and is then switched off: a loop that runs 2 million times took 0.036 s without recording and 0.040 s with it.

Opt-in

• Capability coverage, default false, so existing hosts are unchanged.
• The embed switches it on only when coverage is listed in ?features=. A missing features (which otherwise means "everything") keeps it off.
• No protocol change: features is a query parameter.

Checks

• scripts/check-runtime-coverage.py (new, in test:runtime):
  • the three reference cases (ist_genuegend fully covered, bewertung → line 11, vorzeichen → line 12)
  • a library file, top-level code, a function without examples, grouping, statements over several lines, global/while True/nested def, try/methods/decorators/lambdas, a failing example
  • every case is also checked against Python's trace module
• check-console-browser.mjs, in Chrome with Pyodide:
  • the report for bewertung and ist_genuegend (which also shows that sys.monitoring works in Pyodide)
  • nothing with a failing example
  • nothing without coverage in features, and nothing without features

Lint (known warning only), both builds, test:protocol, test:hints, test:runtime and test:console pass locally. The IDE was checked locally with this build: marks on the right lines, cleared on edit.

After the merge: tag v0.1.17, then the IDE PR (bump + coverage: true) and the macro PR (features=coverage).

🤖 Generated with Claude Code